2026-05-06 07:31:01,694 [ 205411 ] INFO : ClickHouse root is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse (runner:42, check_args_and_update_paths) 2026-05-06 07:31:01,694 [ 205411 ] INFO : Cases dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:86, check_args_and_update_paths) 2026-05-06 07:31:01,695 [ 205411 ] INFO : utils dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/utils (runner:97, check_args_and_update_paths) 2026-05-06 07:31:01,695 [ 205411 ] INFO : base_configs_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/programs/server, binary: /home/ubuntu/_work/_temp/test/build/clickhouse, cases_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:99, check_args_and_update_paths) clickhouse_integration_tests_volume Running pytest container as: 'docker run --rm --name clickhouse_integration_tests_g2b5um --privileged --dns-search='.' --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=2cffe1eae894 -e DOCKER_BASE_TAG=1e0b53d756cf -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=caad4729259e -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 test_backup_restore_on_cluster/test_concurrency.py::test_kill_mutation_during_backup test_backup_restore_on_cluster/test_concurrency.py::test_replicated_table test_backward_compatibility/test.py::test_backward_compatability1 test_backward_compatibility/test_aggregate_fixed_key.py::test_two_level_merge test_backward_compatibility/test_convert_ordinary.py::test_convert_ordinary_to_atomic test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability test_backward_compatibility/test_select_aggregate_alias_column.py::test_select_aggregate_alias_column test_backward_compatibility/test_short_strings_aggregation.py::test_backward_compatability test_check_table/test.py::test_check_all_tables 'test_check_table/test.py::test_check_normal_table_corruption[]' 'test_check_table/test.py::test_check_replicated_table_corruption[-_0-.bin]' 'test_check_table/test.py::test_check_replicated_table_simple[-_0]' test_cleanup_dir_after_bad_zk_conn/test.py::test_cleanup_dir_after_wrong_replica_name test_cleanup_dir_after_bad_zk_conn/test.py::test_cleanup_dir_after_wrong_zk_path test_cluster_all_replicas/test.py::test_cluster 'test_cluster_all_replicas/test.py::test_error_on_unavailable_replica[one_shard_three_nodes]' 'test_cluster_all_replicas/test.py::test_error_on_unavailable_replica[two_shards_three_nodes]' test_cluster_all_replicas/test.py::test_global_in 'test_cluster_all_replicas/test.py::test_skip_unavailable_replica[one_shard_three_nodes]' 'test_cluster_all_replicas/test.py::test_skip_unavailable_replica[two_shards_three_nodes]' test_cluster_discovery/test.py::test_cluster_discovery_startup_and_stop test_compression_nested_columns/test.py::test_nested_compression_codec test_concurrent_queries_for_user_restriction/test.py::test_exception_message test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool test_config_corresponding_root/test.py::test_work test_config_decryption/test_wrong_settings.py::test_invalid_chars test_config_decryption/test_wrong_settings.py::test_no_encryption_key test_config_decryption/test_wrong_settings.py::test_subnodes test_config_decryption/test_wrong_settings.py::test_wrong_method test_config_reloader_interval/test.py::test_reload_config test_config_xml_full/test.py::test_xml_full_conf test_config_yaml_merge_keys/test.py::test_yaml_merge_keys_conf test_create_query_constraints/test.py::test_create_query_const_constraints test_create_query_constraints/test.py::test_create_query_minmax_constraints test_create_user_and_login/test.py::test_grant_create_user test_create_user_and_login/test.py::test_login test_create_user_and_login/test.py::test_login_as_dropped_user test_create_user_and_login/test.py::test_login_as_dropped_user_xml test_custom_settings/test.py::test_custom_settings test_custom_settings/test.py::test_illformed_setting test_ddl_config_hostname/test.py::test_ddl_queue_delete_add_replica 'test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_cache]' 'test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_direct]' 'test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_hashed]' 'test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_ranged[range_hashed]' 'test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[cache]' 'test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[direct]' 'test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[flat]' 'test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[hashed]' 'test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_cache]' 'test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_direct]' 'test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_hashed]' 'test_dictionaries_all_layouts_separate_sources/test_http.py::test_ranged[range_hashed]' 'test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[cache]' 'test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[direct]' 'test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[flat]' 'test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[hashed]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[DateTime_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[Date_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[Float32_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[Float64_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[Int16_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[Int32_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[Int64_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[Int8_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[String_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[UInt16_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[UInt32_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[UInt64_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[UInt8_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[UUID_]' 'test_dictionaries_select_all/test.py::test_select_all[clickhouse_complex_integers_key_hashed]' 'test_dictionaries_select_all/test.py::test_select_all[clickhouse_complex_mixed_key_hashed]' 'test_dictionaries_select_all/test.py::test_select_all[clickhouse_flat]' 'test_dictionaries_select_all/test.py::test_select_all[clickhouse_hashed]' 'test_dictionaries_select_all/test.py::test_select_all[clickhouse_range_hashed]' 'test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_cache]' 'test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_complex_integers_key_cache]' 'test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_complex_mixed_key_cache]' test_dictionaries_wait_for_load/test.py::test_wait_for_dictionaries_load test_disable_insertion_and_mutation/test.py::test_disable_insertion_and_mutation test_distributed_async_insert_for_node_changes/test.py::test_distributed_async_insert test_distributed_async_insert_for_node_changes/test.py::test_distributed_async_insert_with_replica 'test_distributed_ddl/test_replicated_alter.py::test_replicated_alters[configs]' 'test_distributed_ddl/test_replicated_alter.py::test_replicated_alters[configs_secure]' test_distributed_ddl_parallel/test.py::test_all_in_parallel test_distributed_ddl_parallel/test.py::test_slow_dict_load_7 test_distributed_ddl_parallel/test.py::test_smoke test_distributed_ddl_parallel/test.py::test_smoke_parallel test_distributed_ddl_parallel/test.py::test_smoke_parallel_dict_reload test_distributed_ddl_parallel/test.py::test_two_in_parallel_two_queued 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[cluster_node1]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[cluster_node2]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[dod_node1]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[dod_node2]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_group_by_key[cluster_node1]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_group_by_key[cluster_node2]' -vvv" altinityinfra/integration-tests-runner:37a9815fd2fa '. Start tests ============================= test session starts ============================== platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /usr/bin/python3 cachedir: .pytest_cache rootdir: /ClickHouse/tests/integration configfile: pytest.ini plugins: order-1.0.1, random-0.2, timeout-2.2.0, repeat-0.9.3, reportlog-0.4.0, xdist-3.5.0 timeout: 900.0s timeout method: signal timeout func_only: False created: 10/10 workers 10 workers [100 items] scheduling tests via LoadFileScheduling test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_cache] test_dictionaries_redis/test.py::test_redis_dictionaries[DateTime_] test_dictionaries_select_all/test.py::test_select_all[clickhouse_complex_integers_key_hashed] test_distributed_ddl_parallel/test.py::test_all_in_parallel test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_cache] test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool test_check_table/test.py::test_check_all_tables test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[cluster_node1] test_config_decryption/test_wrong_settings.py::test_invalid_chars test_cluster_all_replicas/test.py::test_cluster [gw2] [ 1%] PASSED test_dictionaries_select_all/test.py::test_select_all[clickhouse_complex_integers_key_hashed] test_dictionaries_select_all/test.py::test_select_all[clickhouse_complex_mixed_key_hashed] [gw2] [ 2%] PASSED test_dictionaries_select_all/test.py::test_select_all[clickhouse_complex_mixed_key_hashed] test_dictionaries_select_all/test.py::test_select_all[clickhouse_flat] [gw4] [ 3%] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[cluster_node1] test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[cluster_node2] [gw6] [ 4%] PASSED test_cluster_all_replicas/test.py::test_cluster test_cluster_all_replicas/test.py::test_error_on_unavailable_replica[one_shard_three_nodes] [gw2] [ 5%] PASSED test_dictionaries_select_all/test.py::test_select_all[clickhouse_flat] test_dictionaries_select_all/test.py::test_select_all[clickhouse_hashed] [gw6] [ 6%] PASSED test_cluster_all_replicas/test.py::test_error_on_unavailable_replica[one_shard_three_nodes] test_cluster_all_replicas/test.py::test_error_on_unavailable_replica[two_shards_three_nodes] [gw8] [ 7%] PASSED test_config_decryption/test_wrong_settings.py::test_invalid_chars test_config_decryption/test_wrong_settings.py::test_no_encryption_key [gw6] [ 8%] PASSED test_cluster_all_replicas/test.py::test_error_on_unavailable_replica[two_shards_three_nodes] test_cluster_all_replicas/test.py::test_global_in [gw4] [ 9%] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[cluster_node2] test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[dod_node1] [gw2] [ 10%] PASSED test_dictionaries_select_all/test.py::test_select_all[clickhouse_hashed] test_dictionaries_select_all/test.py::test_select_all[clickhouse_range_hashed] [gw4] [ 11%] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[dod_node1] test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[dod_node2] [gw4] [ 12%] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[dod_node2] test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_group_by_key[cluster_node1] [gw6] [ 13%] PASSED test_cluster_all_replicas/test.py::test_global_in test_cluster_all_replicas/test.py::test_skip_unavailable_replica[one_shard_three_nodes] [gw2] [ 14%] PASSED test_dictionaries_select_all/test.py::test_select_all[clickhouse_range_hashed] test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_cache] [gw4] [ 15%] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_group_by_key[cluster_node1] test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_group_by_key[cluster_node2] [gw6] [ 16%] PASSED test_cluster_all_replicas/test.py::test_skip_unavailable_replica[one_shard_three_nodes] test_cluster_all_replicas/test.py::test_skip_unavailable_replica[two_shards_three_nodes] [gw6] [ 17%] PASSED test_cluster_all_replicas/test.py::test_skip_unavailable_replica[two_shards_three_nodes] [gw4] [ 18%] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_group_by_key[cluster_node2] test_create_user_and_login/test.py::test_grant_create_user [gw2] [ 19%] PASSED test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_cache] test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_complex_integers_key_cache] [gw2] [ 20%] PASSED test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_complex_integers_key_cache] test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_complex_mixed_key_cache] test_backup_restore_on_cluster/test_concurrency.py::test_kill_mutation_during_backup [gw4] [ 21%] PASSED test_create_user_and_login/test.py::test_grant_create_user test_create_user_and_login/test.py::test_login [gw8] [ 22%] PASSED test_config_decryption/test_wrong_settings.py::test_no_encryption_key test_config_decryption/test_wrong_settings.py::test_subnodes [gw2] [ 23%] PASSED test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_complex_mixed_key_cache] [gw4] [ 24%] PASSED test_create_user_and_login/test.py::test_login test_create_user_and_login/test.py::test_login_as_dropped_user [gw4] [ 25%] PASSED test_create_user_and_login/test.py::test_login_as_dropped_user test_create_user_and_login/test.py::test_login_as_dropped_user_xml [gw7] [ 26%] PASSED test_check_table/test.py::test_check_all_tables test_check_table/test.py::test_check_normal_table_corruption[] test_cleanup_dir_after_bad_zk_conn/test.py::test_cleanup_dir_after_wrong_replica_name [gw4] [ 27%] PASSED test_create_user_and_login/test.py::test_login_as_dropped_user_xml [gw5] [ 28%] PASSED test_distributed_ddl_parallel/test.py::test_all_in_parallel test_distributed_ddl_parallel/test.py::test_slow_dict_load_7 [gw8] [ 29%] PASSED test_config_decryption/test_wrong_settings.py::test_subnodes test_config_decryption/test_wrong_settings.py::test_wrong_method [gw7] [ 30%] PASSED test_check_table/test.py::test_check_normal_table_corruption[] test_check_table/test.py::test_check_replicated_table_corruption[-_0-.bin] test_distributed_async_insert_for_node_changes/test.py::test_distributed_async_insert [gw2] [ 31%] PASSED test_cleanup_dir_after_bad_zk_conn/test.py::test_cleanup_dir_after_wrong_replica_name test_cleanup_dir_after_bad_zk_conn/test.py::test_cleanup_dir_after_wrong_zk_path [gw1] [ 32%] PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[DateTime_] test_dictionaries_redis/test.py::test_redis_dictionaries[Date_] [gw5] [ 33%] PASSED test_distributed_ddl_parallel/test.py::test_slow_dict_load_7 test_distributed_ddl_parallel/test.py::test_smoke [gw7] [ 34%] PASSED test_check_table/test.py::test_check_replicated_table_corruption[-_0-.bin] test_check_table/test.py::test_check_replicated_table_simple[-_0] [gw2] [ 35%] PASSED test_cleanup_dir_after_bad_zk_conn/test.py::test_cleanup_dir_after_wrong_zk_path [gw9] [ 36%] PASSED test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated [gw8] [ 37%] PASSED test_config_decryption/test_wrong_settings.py::test_wrong_method test_custom_settings/test.py::test_custom_settings [gw3] [ 38%] PASSED test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_cache] test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_direct] [gw7] [ 39%] PASSED test_check_table/test.py::test_check_replicated_table_simple[-_0] [gw0] [ 40%] PASSED test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_cache] test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_direct] [gw4] [ 41%] PASSED test_distributed_async_insert_for_node_changes/test.py::test_distributed_async_insert test_distributed_async_insert_for_node_changes/test.py::test_distributed_async_insert_with_replica test_distributed_ddl/test_replicated_alter.py::test_replicated_alters[configs] [gw8] [ 42%] PASSED test_custom_settings/test.py::test_custom_settings test_custom_settings/test.py::test_illformed_setting [gw8] [ 43%] PASSED test_custom_settings/test.py::test_illformed_setting [gw4] [ 44%] PASSED test_distributed_async_insert_for_node_changes/test.py::test_distributed_async_insert_with_replica test_backward_compatibility/test.py::test_backward_compatability1 test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability [gw1] [ 45%] PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[Date_] test_dictionaries_redis/test.py::test_redis_dictionaries[Float32_] test_backward_compatibility/test_aggregate_fixed_key.py::test_two_level_merge [gw6] [ 46%] PASSED test_backup_restore_on_cluster/test_concurrency.py::test_kill_mutation_during_backup test_backup_restore_on_cluster/test_concurrency.py::test_replicated_table [gw5] [ 47%] PASSED test_distributed_ddl_parallel/test.py::test_smoke test_distributed_ddl_parallel/test.py::test_smoke_parallel [gw5] [ 48%] PASSED test_distributed_ddl_parallel/test.py::test_smoke_parallel test_distributed_ddl_parallel/test.py::test_smoke_parallel_dict_reload [gw6] [ 49%] PASSED test_backup_restore_on_cluster/test_concurrency.py::test_replicated_table [gw7] [ 50%] PASSED test_backward_compatibility/test.py::test_backward_compatability1 [gw8] [ 51%] PASSED test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability [gw1] [ 52%] PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[Float32_] test_dictionaries_redis/test.py::test_redis_dictionaries[Float64_] [gw4] [ 53%] PASSED test_backward_compatibility/test_aggregate_fixed_key.py::test_two_level_merge [gw5] [ 54%] PASSED test_distributed_ddl_parallel/test.py::test_smoke_parallel_dict_reload test_distributed_ddl_parallel/test.py::test_two_in_parallel_two_queued test_cluster_discovery/test.py::test_cluster_discovery_startup_and_stop [gw3] [ 55%] PASSED test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_direct] test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_hashed] test_backward_compatibility/test_convert_ordinary.py::test_convert_ordinary_to_atomic [gw0] [ 56%] PASSED test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_direct] test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_hashed] [gw2] [ 57%] PASSED test_distributed_ddl/test_replicated_alter.py::test_replicated_alters[configs] test_distributed_ddl/test_replicated_alter.py::test_replicated_alters[configs_secure] test_backward_compatibility/test_select_aggregate_alias_column.py::test_select_aggregate_alias_column test_create_query_constraints/test.py::test_create_query_const_constraints [gw5] [ 58%] PASSED test_distributed_ddl_parallel/test.py::test_two_in_parallel_two_queued [gw1] [ 59%] PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[Float64_] test_dictionaries_redis/test.py::test_redis_dictionaries[Int16_] [gw4] [ 60%] PASSED test_backward_compatibility/test_select_aggregate_alias_column.py::test_select_aggregate_alias_column [gw6] [ 61%] PASSED test_create_query_constraints/test.py::test_create_query_const_constraints test_create_query_constraints/test.py::test_create_query_minmax_constraints [gw6] [ 62%] PASSED test_create_query_constraints/test.py::test_create_query_minmax_constraints test_config_corresponding_root/test.py::test_work test_config_reloader_interval/test.py::test_reload_config [gw4] [ 63%] PASSED test_config_corresponding_root/test.py::test_work test_disable_insertion_and_mutation/test.py::test_disable_insertion_and_mutation [gw5] [ 64%] PASSED test_config_reloader_interval/test.py::test_reload_config [gw9] [ 65%] PASSED test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas [gw1] [ 66%] PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[Int16_] test_dictionaries_redis/test.py::test_redis_dictionaries[Int32_] test_config_xml_full/test.py::test_xml_full_conf [gw0] [ 67%] PASSED test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_hashed] test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_ranged[range_hashed] [gw3] [ 68%] PASSED test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_hashed] test_dictionaries_all_layouts_separate_sources/test_http.py::test_ranged[range_hashed] [gw2] [ 69%] PASSED test_distributed_ddl/test_replicated_alter.py::test_replicated_alters[configs_secure] [gw0] [ 70%] PASSED test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_ranged[range_hashed] test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[cache] [gw1] [ 71%] PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[Int32_] test_dictionaries_redis/test.py::test_redis_dictionaries[Int64_] [gw5] [ 72%] PASSED test_config_xml_full/test.py::test_xml_full_conf [gw3] [ 73%] PASSED test_dictionaries_all_layouts_separate_sources/test_http.py::test_ranged[range_hashed] test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[cache] [gw4] [ 74%] PASSED test_disable_insertion_and_mutation/test.py::test_disable_insertion_and_mutation test_backward_compatibility/test_short_strings_aggregation.py::test_backward_compatability [gw1] [ 75%] PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[Int64_] test_dictionaries_redis/test.py::test_redis_dictionaries[Int8_] [gw2] [ 76%] PASSED test_backward_compatibility/test_short_strings_aggregation.py::test_backward_compatability [gw8] [ 77%] FAILED test_cluster_discovery/test.py::test_cluster_discovery_startup_and_stop test_dictionaries_wait_for_load/test.py::test_wait_for_dictionaries_load test_concurrent_queries_for_user_restriction/test.py::test_exception_message [gw0] [ 78%] PASSED test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[cache] test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[direct] [gw3] [ 79%] PASSED test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[cache] test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[direct] [gw1] [ 80%] PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[Int8_] test_dictionaries_redis/test.py::test_redis_dictionaries[String_] [gw8] [ 81%] PASSED test_concurrent_queries_for_user_restriction/test.py::test_exception_message test_config_yaml_merge_keys/test.py::test_yaml_merge_keys_conf [gw1] [ 82%] PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[String_] test_dictionaries_redis/test.py::test_redis_dictionaries[UInt16_] [gw2] [ 83%] PASSED test_dictionaries_wait_for_load/test.py::test_wait_for_dictionaries_load [gw8] [ 84%] PASSED test_config_yaml_merge_keys/test.py::test_yaml_merge_keys_conf [gw9] [ 85%] PASSED test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool [gw0] [ 86%] PASSED test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[direct] test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[flat] [gw1] [ 87%] PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[UInt16_] test_dictionaries_redis/test.py::test_redis_dictionaries[UInt32_] [gw3] [ 88%] PASSED test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[direct] test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[flat] [gw7] [ 89%] PASSED test_backward_compatibility/test_convert_ordinary.py::test_convert_ordinary_to_atomic [gw9] [ 90%] PASSED test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool test_compression_nested_columns/test.py::test_nested_compression_codec [gw1] [ 91%] PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[UInt32_] test_dictionaries_redis/test.py::test_redis_dictionaries[UInt64_] [gw0] [ 92%] PASSED test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[flat] test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[hashed] [gw7] [ 93%] PASSED test_compression_nested_columns/test.py::test_nested_compression_codec [gw1] [ 94%] PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[UInt64_] test_dictionaries_redis/test.py::test_redis_dictionaries[UInt8_] [gw3] [ 95%] PASSED test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[flat] test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[hashed] test_ddl_config_hostname/test.py::test_ddl_queue_delete_add_replica [gw1] [ 96%] PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[UInt8_] test_dictionaries_redis/test.py::test_redis_dictionaries[UUID_] [gw7] [ 97%] PASSED test_ddl_config_hostname/test.py::test_ddl_queue_delete_add_replica [gw0] [ 98%] PASSED test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[hashed] [gw3] [ 99%] PASSED test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[hashed] [gw1] [100%] PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[UUID_] =================================== FAILURES =================================== ___________________ test_cluster_discovery_startup_and_stop ____________________ [gw8] linux -- Python 3.10.12 /usr/bin/python3 start_cluster = def test_cluster_discovery_startup_and_stop(start_cluster): """ Start cluster, check nodes count in system.clusters, then stop/start some nodes and check that it (dis)appeared in cluster. """ check_nodes_count = functools.partial( check_on_cluster, what="count()", msg="Wrong nodes count in cluster" ) check_shard_num = functools.partial( check_on_cluster, what="count(DISTINCT shard_num)", msg="Wrong shard_num count in cluster", ) # `- 1` because one node is an observer total_shards = len(set(shard_configs.values())) - 1 total_nodes = len(nodes) - 1 check_nodes_count( [nodes["node0"], nodes["node2"], nodes["node_observer"]], total_nodes ) check_shard_num( [nodes["node0"], nodes["node2"], nodes["node_observer"]], total_shards ) # test ON CLUSTER query nodes["node0"].query( "CREATE TABLE tbl ON CLUSTER 'test_auto_cluster' (x UInt64) ENGINE = MergeTree ORDER BY x" ) nodes["node0"].query("INSERT INTO tbl VALUES (1)") nodes["node1"].query("INSERT INTO tbl VALUES (2)") assert ( int( nodes["node_observer"] .query( "SELECT sum(x) FROM clusterAllReplicas(test_auto_cluster, default.tbl)" ) .strip() ) == 3 ) # Query SYSTEM DROP DNS CACHE may reload cluster configuration # check that it does not affect cluster discovery nodes["node1"].query("SYSTEM DROP DNS CACHE") nodes["node0"].query("SYSTEM DROP DNS CACHE") check_shard_num( [nodes["node0"], nodes["node2"], nodes["node_observer"]], total_shards ) nodes["node1"].stop_clickhouse(kill=True) check_nodes_count( [nodes["node0"], nodes["node2"], nodes["node_observer"]], total_nodes - 1 ) # node1 was the only node in shard '1' check_shard_num( [nodes["node0"], nodes["node2"], nodes["node_observer"]], total_shards - 1 ) nodes["node3"].stop_clickhouse() > check_nodes_count( [nodes["node0"], nodes["node2"], nodes["node_observer"]], total_nodes - 2 ) test_cluster_discovery/test.py:104: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ nodes = [, , ] expected = 3 def check_on_cluster( nodes, expected, *, what, cluster_name="test_auto_cluster", msg=None, retries=5, query_params={}, ): """ Select data from `system.clusters` on specified nodes and check the result """ assert 1 <= retries <= 6 node_results = {} for retry in range(1, retries + 1): for node in nodes: if node_results.get(node.name) == expected: # do not retry node after success continue query_text = ( f"SELECT {what} FROM system.clusters WHERE cluster = '{cluster_name}'" ) node_results[node.name] = int(node.query(query_text, **query_params)) if all(actual == expected for actual in node_results.values()): break print(f"Retry {retry}/{retries} unsuccessful, result: {node_results}") if retry != retries: time.sleep(2**retry) else: msg = msg or f"Wrong '{what}' result" > raise Exception( f"{msg}: {node_results}, expected: {expected} (after {retries} retries)" ) E Exception: Wrong nodes count in cluster: {'node0': 3, 'node2': 4, 'node_observer': 3}, expected: 3 (after 5 retries) test_cluster_discovery/common.py:39: Exception ---------------------------- Captured stdout setup ----------------------------- Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml ------------------------------ Captured log setup ------------------------------ 2026-05-06 07:33:13 [ 563 ] INFO : Running tests in /ClickHouse/tests/integration/test_cluster_discovery/test.py (cluster.py:2788, start) 2026-05-06 07:33:13 [ 563 ] DEBUG : Cluster start called. is_up=False (cluster.py:2795, start) 2026-05-06 07:33:13 [ 563 ] DEBUG : Docker networks for project roottestclusterdiscovery_gw8 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-05-06 07:33:13 [ 563 ] DEBUG : Docker containers for project roottestclusterdiscovery_gw8 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-05-06 07:33:13 [ 563 ] DEBUG : Docker volumes for project roottestclusterdiscovery_gw8 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-05-06 07:33:13 [ 563 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-05-06 07:33:13 [ 563 ] DEBUG : Docker networks for project roottestclusterdiscovery_gw8 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-05-06 07:33:14 [ 563 ] DEBUG : Docker containers for project roottestclusterdiscovery_gw8 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-05-06 07:33:14 [ 563 ] DEBUG : Docker volumes for project roottestclusterdiscovery_gw8 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-05-06 07:33:14 [ 563 ] DEBUG : Command:docker container list --all --filter name='^/roottestclusterdiscovery_gw8_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-05-06 07:33:14 [ 563 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-05-06 07:33:14 [ 563 ] DEBUG : No running containers for project: roottestclusterdiscovery_gw8 (cluster.py:904, cleanup) 2026-05-06 07:33:14 [ 563 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-05-06 07:33:14 [ 563 ] DEBUG : Trying to prune unused images... (cluster.py:926, cleanup) 2026-05-06 07:33:14 [ 563 ] DEBUG : Command:['docker', 'image', 'prune', '-f'] (cluster.py:113, run_and_check) 2026-05-06 07:33:14 [ 563 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:121, run_and_check) 2026-05-06 07:33:14 [ 563 ] DEBUG : Images pruned (cluster.py:929, cleanup) 2026-05-06 07:33:14 [ 563 ] DEBUG : Trying to prune unused volumes... (cluster.py:935, cleanup) 2026-05-06 07:33:14 [ 563 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-05-06 07:33:14 [ 563 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-05-06 07:33:14 [ 563 ] DEBUG : Setup directory for instance: node0 (cluster.py:2808, start) 2026-05-06 07:33:14 [ 563 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4534, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Create directory for common tests configuration (cluster.py:4539, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Copy common configuration from helpers (cluster.py:4559, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Generate and write macros file (cluster.py:4602, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_cluster_discovery/config/config.xml'] to /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node0/configs/config.d (cluster.py:4632, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node0/database (cluster.py:4649, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node0/logs (cluster.py:4660, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4746, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Setup directory for instance: node1 (cluster.py:2808, start) 2026-05-06 07:33:14 [ 563 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4534, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Create directory for common tests configuration (cluster.py:4539, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Copy common configuration from helpers (cluster.py:4559, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Generate and write macros file (cluster.py:4602, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_cluster_discovery/config/config_shard1.xml'] to /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node1/configs/config.d (cluster.py:4632, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node1/database (cluster.py:4649, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node1/logs (cluster.py:4660, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4746, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Setup directory for instance: node2 (cluster.py:2808, start) 2026-05-06 07:33:14 [ 563 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4534, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Create directory for common tests configuration (cluster.py:4539, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Copy common configuration from helpers (cluster.py:4559, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Generate and write macros file (cluster.py:4602, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_cluster_discovery/config/config.xml'] to /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node2/configs/config.d (cluster.py:4632, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node2/database (cluster.py:4649, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node2/logs (cluster.py:4660, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4746, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Setup directory for instance: node3 (cluster.py:2808, start) 2026-05-06 07:33:14 [ 563 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4534, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Create directory for common tests configuration (cluster.py:4539, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Copy common configuration from helpers (cluster.py:4559, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Generate and write macros file (cluster.py:4602, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_cluster_discovery/config/config_shard3.xml'] to /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node3/configs/config.d (cluster.py:4632, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node3/database (cluster.py:4649, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node3/logs (cluster.py:4660, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4746, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Setup directory for instance: node4 (cluster.py:2808, start) 2026-05-06 07:33:14 [ 563 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4534, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Create directory for common tests configuration (cluster.py:4539, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Copy common configuration from helpers (cluster.py:4559, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Generate and write macros file (cluster.py:4602, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_cluster_discovery/config/config.xml'] to /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node4/configs/config.d (cluster.py:4632, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node4/database (cluster.py:4649, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node4/logs (cluster.py:4660, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4746, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Setup directory for instance: node_observer (cluster.py:2808, start) 2026-05-06 07:33:14 [ 563 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4534, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Create directory for common tests configuration (cluster.py:4539, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Copy common configuration from helpers (cluster.py:4559, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Generate and write macros file (cluster.py:4602, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_cluster_discovery/config/config_observer.xml'] to /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node_observer/configs/config.d (cluster.py:4632, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node_observer/database (cluster.py:4649, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node_observer/logs (cluster.py:4660, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4746, create_dir) 2026-05-06 07:33:14 [ 563 ] DEBUG : Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:1e0b53d756cf', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper3/coordination'} stored in /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/.env (cluster.py:86, _create_env_file) 2026-05-06 07:33:14 [ 563 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-05-06 07:33:14 [ 563 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-05-06 07:33:14 [ 563 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-05-06 07:33:14 [ 563 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-05-06 07:33:14 [ 563 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 824 (connectionpool.py:547, _make_request) 2026-05-06 07:33:14 [ 563 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/.env', '--project-name', 'roottestclusterdiscovery_gw8', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node0/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node1/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node3/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node4/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node_observer/docker-compose.yml', 'pull'] (cluster.py:113, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling zoo3 ... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling zoo1 ... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling zoo2 ... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node0 ... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node1 ... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node2 ... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node4 ... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node_observer ... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node3 ... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling zoo3 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node_observer ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling zoo3 ... digest: sha256:bf725030a292d5daab... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling zoo3 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling zoo3 ... done (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node_observer ... digest: sha256:bf725030a292d5daab... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node_observer ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node_observer ... done (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node1 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node2 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling zoo1 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node1 ... digest: sha256:bf725030a292d5daab... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node2 ... digest: sha256:bf725030a292d5daab... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling zoo1 ... digest: sha256:bf725030a292d5daab... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling zoo1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node2 ... done (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling zoo1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node3 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node3 ... digest: sha256:bf725030a292d5daab... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node3 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node3 ... done (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node4 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling zoo2 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node0 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node4 ... digest: sha256:bf725030a292d5daab... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node4 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node4 ... done (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling zoo2 ... digest: sha256:bf725030a292d5daab... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling zoo2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling zoo2 ... done (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node0 ... digest: sha256:bf725030a292d5daab... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node0 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Stderr:Pulling node0 ... done (cluster.py:123, run_and_check) 2026-05-06 07:33:36 [ 563 ] DEBUG : Setup ZooKeeper (cluster.py:2849, start) 2026-05-06 07:33:36 [ 563 ] DEBUG : Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper1/log', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper1/config', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper1/coordination', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper2/log', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper2/config', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper2/coordination', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper3/log', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper3/config', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper3/coordination'] (cluster.py:2850, start) 2026-05-06 07:33:36 [ 563 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/.env', '--project-name', 'roottestclusterdiscovery_gw8', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--verbose', 'up', '-d'] (cluster.py:113, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.config.config.find: Using configuration files: /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.docker_client.get_client: docker-compose version 1.29.2, build unknown (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:docker-py version: (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:CPython version: 3.10.12 (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:OpenSSL version: OpenSSL 3.0.2 15 Mar 2022 (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker base_url: http+docker://localhost (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker version: Platform={'Name': 'Docker Engine - Community'}, Components=[{'Name': 'Engine', 'Version': '23.0.6', 'Details': {'ApiVersion': '1.42', 'Arch': 'amd64', 'BuildTime': '2023-05-05T21:18:13.000000000+00:00', 'Experimental': 'false', 'GitCommit': '9dbdbd4', 'GoVersion': 'go1.19.9', 'KernelVersion': '5.15.0-130-generic', 'MinAPIVersion': '1.12', 'Os': 'linux'}}, {'Name': 'containerd', 'Version': '1.7.25', 'Details': {'GitCommit': 'bcc810d6b9066471b0b6fa75f557a15a1cbf31bb'}}, {'Name': 'runc', 'Version': '1.2.4', 'Details': {'GitCommit': 'v1.2.4-0-g6c52b3f'}}, {'Name': 'docker-init', 'Version': '0.19.0', 'Details': {'GitCommit': 'de40ad0'}}], Version=23.0.6, ApiVersion=1.42, MinAPIVersion=1.12, GitCommit=9dbdbd4, GoVersion=go1.19.9, Os=linux, Arch=amd64, KernelVersion=5.15.0-130-generic, BuildTime=2023-05-05T21:18:13.000000000+00:00 (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('roottestclusterdiscoverygw8_default') (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info <- () (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info -> {'Architecture': 'x86_64', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'BridgeNfIp6tables': True, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'BridgeNfIptables': True, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'CPUSet': True, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'CPUShares': True, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'CgroupDriver': 'cgroupfs', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'CgroupVersion': '2', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'ContainerdCommit': {'Expected': 'bcc810d6b9066471b0b6fa75f557a15a1cbf31bb', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'ID': 'bcc810d6b9066471b0b6fa75f557a15a1cbf31bb'}, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Containers': 27, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('roottestclusterdiscovery_gw8_default') (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.network.ensure: Creating network "roottestclusterdiscovery_gw8_default" with the default driver (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_network <- (name='roottestclusterdiscovery_gw8_default', driver=None, options=None, ipam=None, internal=False, enable_ipv6=False, labels={'com.docker.compose.project': 'roottestclusterdiscovery_gw8', 'com.docker.compose.network': 'default', 'com.docker.compose.version': '1.29.2'}, attachable=True, check_duplicate=True) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_network -> {'Id': 'a238e61a61420a63a83cf3decf41f043c55e25153f9f246ff8e2e76d2952abef', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Warning': ''} (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=roottestclusterdiscovery_gw8', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=roottestclusterdiscoverygw8', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestclusterdiscovery_gw8', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestclusterdiscoverygw8', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestclusterdiscovery_gw8', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestclusterdiscoverygw8', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestclusterdiscovery_gw8', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestclusterdiscoverygw8', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestclusterdiscovery_gw8', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestclusterdiscoverygw8', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:1e0b53d756cf') (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:1e0b53d756cf') (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:1e0b53d756cf') (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestclusterdiscovery_gw8', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestclusterdiscoverygw8', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestclusterdiscovery_gw8', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestclusterdiscoverygw8', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestclusterdiscovery_gw8', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestclusterdiscoverygw8', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, , } (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestclusterdiscovery_gw8', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestclusterdiscovery_gw8', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestclusterdiscoverygw8', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestclusterdiscovery_gw8', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:Creating roottestclusterdiscovery_gw8_zoo1_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestclusterdiscoverygw8', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='roottestclusterdiscovery_gw8', service='zoo1', number=1)} (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='roottestclusterdiscovery_gw8', service='zoo1', number=1) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:Creating roottestclusterdiscovery_gw8_zoo3_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestclusterdiscoverygw8', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:1e0b53d756cf') (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='roottestclusterdiscovery_gw8', service='zoo3', number=1)} (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:Creating roottestclusterdiscovery_gw8_zoo2_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='roottestclusterdiscovery_gw8', service='zoo3', number=1) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='roottestclusterdiscovery_gw8', service='zoo2', number=1)} (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:1e0b53d756cf') (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='roottestclusterdiscovery_gw8', service='zoo2', number=1) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:1e0b53d756cf') (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:1e0b53d756cf') (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:1e0b53d756cf') (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: b8c8588e21b9e171db1681963024ba1545d158110944a30b3cf47f82448385a0 (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='roottestclusterdiscovery_gw8_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper1/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper1/log', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper1/coordination', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Target': '/var/lib/clickhouse-keeper', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Type': 'bind'}, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: {'ReadOnly': None, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config1.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:1e0b53d756cf', user='0', volumes={}, name='roottestclusterdiscovery_gw8_zoo1_1', detach=True, environment=[], labels={'com.docker.compose.project': 'roottestclusterdiscovery_gw8', 'com.docker.compose.service': 'zoo1', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': 'b8c8588e21b9e171db1681963024ba1545d158110944a30b3cf47f82448385a0'}, host_config={'NetworkMode': 'roottestclusterdiscovery_gw8_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper1/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper1/log', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'roottestclusterdiscovery_gw8_default': {'Aliases': ['zoo1'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: f2c07d91867223b561b16dfff65b305e747f3ea7380d6dc281b42daa393b72e3 (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='roottestclusterdiscovery_gw8_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper3/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper3/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper3/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper3/log', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Target': '/var/log/clickhouse-keeper', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Type': 'bind'}, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: {'ReadOnly': None, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config3.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:1e0b53d756cf', user='0', volumes={}, name='roottestclusterdiscovery_gw8_zoo3_1', detach=True, environment=[], labels={'com.docker.compose.project': 'roottestclusterdiscovery_gw8', 'com.docker.compose.service': 'zoo3', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': 'f2c07d91867223b561b16dfff65b305e747f3ea7380d6dc281b42daa393b72e3'}, host_config={'NetworkMode': 'roottestclusterdiscovery_gw8_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper3/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper3/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper3/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'roottestclusterdiscovery_gw8_default': {'Aliases': ['zoo3'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:1e0b53d756cf') (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 8eba76252ae76165a6659f4d53b1bb74f883441ff7ba2833a0392d1c9069ed6c (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='roottestclusterdiscovery_gw8_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper2/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper2/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper2/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Source': '/clickhouse', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Target': '/usr/bin/clickhouse-keeper', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Type': 'bind'}, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: {'ReadOnly': None, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config2.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:1e0b53d756cf', user='0', volumes={}, name='roottestclusterdiscovery_gw8_zoo2_1', detach=True, environment=[], labels={'com.docker.compose.project': 'roottestclusterdiscovery_gw8', 'com.docker.compose.service': 'zoo2', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '8eba76252ae76165a6659f4d53b1bb74f883441ff7ba2833a0392d1c9069ed6c'}, host_config={'NetworkMode': 'roottestclusterdiscovery_gw8_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper2/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper2/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/keeper2/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'roottestclusterdiscovery_gw8_default': {'Aliases': ['zoo2'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '11a90c16bf645cac4efbe0cbbcec982519c4b8d1b749b3cc88b2e02b652c2f7b', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('11a90c16bf645cac4efbe0cbbcec982519c4b8d1b749b3cc88b2e02b652c2f7b') (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config1.xml', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('11a90c16bf645cac4efbe0cbbcec982519c4b8d1b749b3cc88b2e02b652c2f7b', 'roottestclusterdiscovery_gw8_default') (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '948cb7ef00512de4e77730e506469ef65d1437f009b096e3cd18b7e9609a6f72', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '780029f81d401df138b9e6a1bc7622365c697d654cc7506bf2c3e1ff259180b4', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('780029f81d401df138b9e6a1bc7622365c697d654cc7506bf2c3e1ff259180b4') (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('948cb7ef00512de4e77730e506469ef65d1437f009b096e3cd18b7e9609a6f72') (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config2.xml', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('780029f81d401df138b9e6a1bc7622365c697d654cc7506bf2c3e1ff259180b4', 'roottestclusterdiscovery_gw8_default') (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('11a90c16bf645cac4efbe0cbbcec982519c4b8d1b749b3cc88b2e02b652c2f7b', 'roottestclusterdiscovery_gw8_default', aliases=['11a90c16bf64', 'zoo1'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config3.xml', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('948cb7ef00512de4e77730e506469ef65d1437f009b096e3cd18b7e9609a6f72', 'roottestclusterdiscovery_gw8_default') (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('11a90c16bf645cac4efbe0cbbcec982519c4b8d1b749b3cc88b2e02b652c2f7b') (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('780029f81d401df138b9e6a1bc7622365c697d654cc7506bf2c3e1ff259180b4', 'roottestclusterdiscovery_gw8_default', aliases=['780029f81d40', 'zoo2'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('948cb7ef00512de4e77730e506469ef65d1437f009b096e3cd18b7e9609a6f72', 'roottestclusterdiscovery_gw8_default', aliases=['zoo3', '948cb7ef0051'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('948cb7ef00512de4e77730e506469ef65d1437f009b096e3cd18b7e9609a6f72') (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('780029f81d401df138b9e6a1bc7622365c697d654cc7506bf2c3e1ff259180b4') (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='roottestclusterdiscovery_gw8', service='zoo1', number=1) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:Creating roottestclusterdiscovery_gw8_zoo1_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='roottestclusterdiscovery_gw8', service='zoo2', number=1) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:Creating roottestclusterdiscovery_gw8_zoo2_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='roottestclusterdiscovery_gw8', service='zoo3', number=1) (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:Creating roottestclusterdiscovery_gw8_zoo3_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-05-06 07:33:38 [ 563 ] DEBUG : Wait ZooKeeper to start (cluster.py:2504, wait_zookeeper_to_start) 2026-05-06 07:33:38 [ 563 ] DEBUG : get_instance_ip instance_name=zoo1 (cluster.py:2135, get_instance_ip) 2026-05-06 07:33:38 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestclusterdiscovery_gw8_zoo1_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:38 [ 563 ] DEBUG : get_kazoo_client: zoo1, ip:172.16.26.2, port:2181, use_ssl:False (cluster.py:3286, get_kazoo_client) 2026-05-06 07:33:38 [ 563 ] INFO : Connecting to 172.16.26.2(172.16.26.2):2181, use_ssl: False (connection.py:650, _connect) 2026-05-06 07:33:38 [ 563 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-05-06 07:33:38 [ 563 ] INFO : Connecting to 172.16.26.2(172.16.26.2):2181, use_ssl: False (connection.py:650, _connect) 2026-05-06 07:33:38 [ 563 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-05-06 07:33:39 [ 563 ] INFO : Connecting to 172.16.26.2(172.16.26.2):2181, use_ssl: False (connection.py:650, _connect) 2026-05-06 07:33:39 [ 563 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-05-06 07:33:39 [ 563 ] INFO : Connecting to 172.16.26.2(172.16.26.2):2181, use_ssl: False (connection.py:650, _connect) 2026-05-06 07:33:39 [ 563 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-05-06 07:33:40 [ 563 ] INFO : Connecting to 172.16.26.2(172.16.26.2):2181, use_ssl: False (connection.py:650, _connect) 2026-05-06 07:33:40 [ 563 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-05-06 07:33:41 [ 563 ] INFO : Connecting to 172.16.26.2(172.16.26.2):2181, use_ssl: False (connection.py:650, _connect) 2026-05-06 07:33:41 [ 563 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-05-06 07:33:41 [ 563 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-05-06 07:33:41 [ 563 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-05-06 07:33:41 [ 563 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-05-06 07:33:41 [ 563 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-05-06 07:33:41 [ 563 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-05-06 07:33:41 [ 563 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-05-06 07:33:41 [ 563 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-05-06 07:33:41 [ 563 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-05-06 07:33:41 [ 563 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-05-06 07:33:41 [ 563 ] DEBUG : get_instance_ip instance_name=zoo2 (cluster.py:2135, get_instance_ip) 2026-05-06 07:33:41 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestclusterdiscovery_gw8_zoo2_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:41 [ 563 ] DEBUG : get_kazoo_client: zoo2, ip:172.16.26.4, port:2181, use_ssl:False (cluster.py:3286, get_kazoo_client) 2026-05-06 07:33:41 [ 563 ] INFO : Connecting to 172.16.26.4(172.16.26.4):2181, use_ssl: False (connection.py:650, _connect) 2026-05-06 07:33:41 [ 563 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-05-06 07:33:41 [ 563 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-05-06 07:33:41 [ 563 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-05-06 07:33:41 [ 563 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-05-06 07:33:41 [ 563 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-05-06 07:33:41 [ 563 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-05-06 07:33:41 [ 563 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-05-06 07:33:41 [ 563 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-05-06 07:33:42 [ 563 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-05-06 07:33:42 [ 563 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-05-06 07:33:42 [ 563 ] DEBUG : get_instance_ip instance_name=zoo3 (cluster.py:2135, get_instance_ip) 2026-05-06 07:33:42 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestclusterdiscovery_gw8_zoo3_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:42 [ 563 ] DEBUG : get_kazoo_client: zoo3, ip:172.16.26.3, port:2181, use_ssl:False (cluster.py:3286, get_kazoo_client) 2026-05-06 07:33:42 [ 563 ] INFO : Connecting to 172.16.26.3(172.16.26.3):2181, use_ssl: False (connection.py:650, _connect) 2026-05-06 07:33:42 [ 563 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-05-06 07:33:42 [ 563 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-05-06 07:33:42 [ 563 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-05-06 07:33:42 [ 563 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-05-06 07:33:42 [ 563 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-05-06 07:33:42 [ 563 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-05-06 07:33:42 [ 563 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-05-06 07:33:42 [ 563 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-05-06 07:33:42 [ 563 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-05-06 07:33:42 [ 563 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-05-06 07:33:42 [ 563 ] DEBUG : All instances of ZooKeeper Secure started (cluster.py:2519, wait_zookeeper_nodes_to_start) 2026-05-06 07:33:42 [ 563 ] DEBUG : ('Trying to create ClickHouse instance by command %s', 'docker-compose --env-file /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/.env --project-name roottestclusterdiscovery_gw8 --file /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node0/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node4/docker-compose.yml --file /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node_observer/docker-compose.yml up -d --no-recreate') (cluster.py:3146, start) 2026-05-06 07:33:42 [ 563 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/.env', '--project-name', 'roottestclusterdiscovery_gw8', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node0/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node1/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node3/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node4/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node_observer/docker-compose.yml', 'up', '-d', '--no-recreate'] (cluster.py:113, run_and_check) 2026-05-06 07:33:44 [ 563 ] DEBUG : Stderr:Creating roottestclusterdiscovery_gw8_node3_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:33:44 [ 563 ] DEBUG : Stderr:Creating roottestclusterdiscovery_gw8_node_observer_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:33:44 [ 563 ] DEBUG : Stderr:Creating roottestclusterdiscovery_gw8_node4_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:33:44 [ 563 ] DEBUG : Stderr:Creating roottestclusterdiscovery_gw8_node0_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:33:44 [ 563 ] DEBUG : Stderr:Creating roottestclusterdiscovery_gw8_node2_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:33:44 [ 563 ] DEBUG : Stderr:Creating roottestclusterdiscovery_gw8_node1_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:33:44 [ 563 ] DEBUG : Stderr:Creating roottestclusterdiscovery_gw8_node3_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:33:44 [ 563 ] DEBUG : Stderr:Creating roottestclusterdiscovery_gw8_node0_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:33:44 [ 563 ] DEBUG : Stderr:Creating roottestclusterdiscovery_gw8_node1_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:33:44 [ 563 ] DEBUG : Stderr:Creating roottestclusterdiscovery_gw8_node4_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:33:44 [ 563 ] DEBUG : Stderr:Creating roottestclusterdiscovery_gw8_node2_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:33:44 [ 563 ] DEBUG : Stderr:Creating roottestclusterdiscovery_gw8_node_observer_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:33:44 [ 563 ] DEBUG : ClickHouse instance created (cluster.py:3154, start) 2026-05-06 07:33:44 [ 563 ] DEBUG : get_instance_ip instance_name=node0 (cluster.py:2135, get_instance_ip) 2026-05-06 07:33:44 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestclusterdiscovery_gw8_node0_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:44 [ 563 ] DEBUG : Waiting for ClickHouse start in node0, ip: 172.16.26.7... (cluster.py:3161, start) 2026-05-06 07:33:44 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestclusterdiscovery_gw8_node0_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:44 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/210a2b6318278c63fd772838f62cd6c04fffb75ae1e5bc3b398564b612187bd9/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:44 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/210a2b6318278c63fd772838f62cd6c04fffb75ae1e5bc3b398564b612187bd9/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:44 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/210a2b6318278c63fd772838f62cd6c04fffb75ae1e5bc3b398564b612187bd9/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:44 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/210a2b6318278c63fd772838f62cd6c04fffb75ae1e5bc3b398564b612187bd9/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:44 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/210a2b6318278c63fd772838f62cd6c04fffb75ae1e5bc3b398564b612187bd9/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:44 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/210a2b6318278c63fd772838f62cd6c04fffb75ae1e5bc3b398564b612187bd9/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/210a2b6318278c63fd772838f62cd6c04fffb75ae1e5bc3b398564b612187bd9/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/210a2b6318278c63fd772838f62cd6c04fffb75ae1e5bc3b398564b612187bd9/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/210a2b6318278c63fd772838f62cd6c04fffb75ae1e5bc3b398564b612187bd9/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/210a2b6318278c63fd772838f62cd6c04fffb75ae1e5bc3b398564b612187bd9/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/210a2b6318278c63fd772838f62cd6c04fffb75ae1e5bc3b398564b612187bd9/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : ClickHouse node0 started (cluster.py:3165, start) 2026-05-06 07:33:45 [ 563 ] DEBUG : get_instance_ip instance_name=node1 (cluster.py:2135, get_instance_ip) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestclusterdiscovery_gw8_node1_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : Waiting for ClickHouse start in node1, ip: 172.16.26.6... (cluster.py:3161, start) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestclusterdiscovery_gw8_node1_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/6dc9272f19bc6e2d3b1076839b3701dcf8860704ef17e1201bdaad1b4d9c4f50/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/6dc9272f19bc6e2d3b1076839b3701dcf8860704ef17e1201bdaad1b4d9c4f50/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/6dc9272f19bc6e2d3b1076839b3701dcf8860704ef17e1201bdaad1b4d9c4f50/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : ClickHouse node1 started (cluster.py:3165, start) 2026-05-06 07:33:45 [ 563 ] DEBUG : get_instance_ip instance_name=node2 (cluster.py:2135, get_instance_ip) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestclusterdiscovery_gw8_node2_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : Waiting for ClickHouse start in node2, ip: 172.16.26.10... (cluster.py:3161, start) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestclusterdiscovery_gw8_node2_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/036d0692a72cd7512535d2414e3751f6fe720eb94e742aac6412a72fe2ec5b2c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/036d0692a72cd7512535d2414e3751f6fe720eb94e742aac6412a72fe2ec5b2c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : ClickHouse node2 started (cluster.py:3165, start) 2026-05-06 07:33:45 [ 563 ] DEBUG : get_instance_ip instance_name=node3 (cluster.py:2135, get_instance_ip) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestclusterdiscovery_gw8_node3_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : Waiting for ClickHouse start in node3, ip: 172.16.26.5... (cluster.py:3161, start) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestclusterdiscovery_gw8_node3_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/7992b1ab0a182b10b6ad3f06a618678e313a631b0f7bf25b6d0c03fc93ead732/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : ClickHouse node3 started (cluster.py:3165, start) 2026-05-06 07:33:45 [ 563 ] DEBUG : get_instance_ip instance_name=node4 (cluster.py:2135, get_instance_ip) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestclusterdiscovery_gw8_node4_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : Waiting for ClickHouse start in node4, ip: 172.16.26.8... (cluster.py:3161, start) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestclusterdiscovery_gw8_node4_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/132f697acd01766952d05f1df1eaead3197324fa7453f2a894f023ef7fd2acc1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : ClickHouse node4 started (cluster.py:3165, start) 2026-05-06 07:33:45 [ 563 ] DEBUG : get_instance_ip instance_name=node_observer (cluster.py:2135, get_instance_ip) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestclusterdiscovery_gw8_node_observer_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : Waiting for ClickHouse start in node_observer, ip: 172.16.26.9... (cluster.py:3161, start) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestclusterdiscovery_gw8_node_observer_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/37adc2a0434c1635a6f05f0294ccbe529f6157b835fc5c1a4d222a0de47233ba/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/37adc2a0434c1635a6f05f0294ccbe529f6157b835fc5c1a4d222a0de47233ba/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:45 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/37adc2a0434c1635a6f05f0294ccbe529f6157b835fc5c1a4d222a0de47233ba/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:46 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/37adc2a0434c1635a6f05f0294ccbe529f6157b835fc5c1a4d222a0de47233ba/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:46 [ 563 ] DEBUG : http://localhost:None "GET /v1.42/containers/37adc2a0434c1635a6f05f0294ccbe529f6157b835fc5c1a4d222a0de47233ba/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-05-06 07:33:46 [ 563 ] DEBUG : ClickHouse node_observer started (cluster.py:3165, start) ----------------------------- Captured stdout call ----------------------------- Retry 1/5 unsuccessful, result: {'node0': 5, 'node2': 5, 'node_observer': 5} Retry 2/5 unsuccessful, result: {'node0': 5, 'node2': 5, 'node_observer': 5} Retry 3/5 unsuccessful, result: {'node0': 5, 'node2': 5, 'node_observer': 5} Retry 1/5 unsuccessful, result: {'node0': 4, 'node2': 4, 'node_observer': 4} Retry 2/5 unsuccessful, result: {'node0': 4, 'node2': 4, 'node_observer': 3} Retry 3/5 unsuccessful, result: {'node0': 3, 'node2': 4, 'node_observer': 3} Retry 4/5 unsuccessful, result: {'node0': 3, 'node2': 4, 'node_observer': 3} Retry 5/5 unsuccessful, result: {'node0': 3, 'node2': 4, 'node_observer': 3} ------------------------------ Captured log call ------------------------------- 2026-05-06 07:33:46 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node0 (cluster.py:3602, query) 2026-05-06 07:33:46 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node2 (cluster.py:3602, query) 2026-05-06 07:33:46 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node_observer (cluster.py:3602, query) 2026-05-06 07:33:47 [ 563 ] DEBUG : Executing query SELECT count(DISTINCT shard_num) FROM system.clusters WHERE cluster = 'test_auto_cluster' on node0 (cluster.py:3602, query) 2026-05-06 07:33:47 [ 563 ] DEBUG : Executing query SELECT count(DISTINCT shard_num) FROM system.clusters WHERE cluster = 'test_auto_cluster' on node2 (cluster.py:3602, query) 2026-05-06 07:33:47 [ 563 ] DEBUG : Executing query SELECT count(DISTINCT shard_num) FROM system.clusters WHERE cluster = 'test_auto_cluster' on node_observer (cluster.py:3602, query) 2026-05-06 07:33:48 [ 563 ] DEBUG : Executing query CREATE TABLE tbl ON CLUSTER 'test_auto_cluster' (x UInt64) ENGINE = MergeTree ORDER BY x on node0 (cluster.py:3602, query) 2026-05-06 07:33:48 [ 563 ] DEBUG : Executing query INSERT INTO tbl VALUES (1) on node0 (cluster.py:3602, query) 2026-05-06 07:33:48 [ 563 ] DEBUG : Executing query INSERT INTO tbl VALUES (2) on node1 (cluster.py:3602, query) 2026-05-06 07:33:49 [ 563 ] DEBUG : Executing query SELECT sum(x) FROM clusterAllReplicas(test_auto_cluster, default.tbl) on node_observer (cluster.py:3602, query) 2026-05-06 07:33:49 [ 563 ] DEBUG : Executing query SYSTEM DROP DNS CACHE on node1 (cluster.py:3602, query) 2026-05-06 07:33:49 [ 563 ] DEBUG : Executing query SYSTEM DROP DNS CACHE on node0 (cluster.py:3602, query) 2026-05-06 07:33:49 [ 563 ] DEBUG : Executing query SELECT count(DISTINCT shard_num) FROM system.clusters WHERE cluster = 'test_auto_cluster' on node0 (cluster.py:3602, query) 2026-05-06 07:33:50 [ 563 ] DEBUG : Executing query SELECT count(DISTINCT shard_num) FROM system.clusters WHERE cluster = 'test_auto_cluster' on node2 (cluster.py:3602, query) 2026-05-06 07:33:50 [ 563 ] DEBUG : Executing query SELECT count(DISTINCT shard_num) FROM system.clusters WHERE cluster = 'test_auto_cluster' on node_observer (cluster.py:3602, query) 2026-05-06 07:33:50 [ 563 ] DEBUG : run container_id:roottestclusterdiscovery_gw8_node1_1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] (cluster.py:2173, exec_in_container) 2026-05-06 07:33:50 [ 563 ] DEBUG : Command:['docker', 'exec', '-u', 'root', 'roottestclusterdiscovery_gw8_node1_1', 'bash', '-c', 'ps -C clickhouse'] (cluster.py:113, run_and_check) 2026-05-06 07:33:50 [ 563 ] DEBUG : Stdout: PID TTY TIME CMD (cluster.py:121, run_and_check) 2026-05-06 07:33:50 [ 563 ] DEBUG : Stdout: 8 ? 00:00:01 clickhouse (cluster.py:121, run_and_check) 2026-05-06 07:33:50 [ 563 ] DEBUG : run container_id:roottestclusterdiscovery_gw8_node1_1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill -9 clickhouse'] (cluster.py:2173, exec_in_container) 2026-05-06 07:33:50 [ 563 ] DEBUG : Command:['docker', 'exec', '-u', 'root', 'roottestclusterdiscovery_gw8_node1_1', 'bash', '-c', 'pkill -9 clickhouse'] (cluster.py:113, run_and_check) 2026-05-06 07:33:50 [ 563 ] DEBUG : run container_id:roottestclusterdiscovery_gw8_node1_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2173, exec_in_container) 2026-05-06 07:33:50 [ 563 ] DEBUG : Command:['docker', 'exec', 'roottestclusterdiscovery_gw8_node1_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:113, run_and_check) 2026-05-06 07:33:50 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node0 (cluster.py:3602, query) 2026-05-06 07:33:51 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node2 (cluster.py:3602, query) 2026-05-06 07:33:51 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node_observer (cluster.py:3602, query) 2026-05-06 07:33:53 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node0 (cluster.py:3602, query) 2026-05-06 07:33:54 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node2 (cluster.py:3602, query) 2026-05-06 07:33:54 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node_observer (cluster.py:3602, query) 2026-05-06 07:33:58 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node0 (cluster.py:3602, query) 2026-05-06 07:33:58 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node2 (cluster.py:3602, query) 2026-05-06 07:33:59 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node_observer (cluster.py:3602, query) 2026-05-06 07:34:07 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node0 (cluster.py:3602, query) 2026-05-06 07:34:08 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node2 (cluster.py:3602, query) 2026-05-06 07:34:08 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node_observer (cluster.py:3602, query) 2026-05-06 07:34:08 [ 563 ] DEBUG : Executing query SELECT count(DISTINCT shard_num) FROM system.clusters WHERE cluster = 'test_auto_cluster' on node0 (cluster.py:3602, query) 2026-05-06 07:34:08 [ 563 ] DEBUG : Executing query SELECT count(DISTINCT shard_num) FROM system.clusters WHERE cluster = 'test_auto_cluster' on node2 (cluster.py:3602, query) 2026-05-06 07:34:12 [ 563 ] DEBUG : Executing query SELECT count(DISTINCT shard_num) FROM system.clusters WHERE cluster = 'test_auto_cluster' on node_observer (cluster.py:3602, query) 2026-05-06 07:34:12 [ 563 ] DEBUG : run container_id:roottestclusterdiscovery_gw8_node3_1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] (cluster.py:2173, exec_in_container) 2026-05-06 07:34:12 [ 563 ] DEBUG : Command:['docker', 'exec', '-u', 'root', 'roottestclusterdiscovery_gw8_node3_1', 'bash', '-c', 'ps -C clickhouse'] (cluster.py:113, run_and_check) 2026-05-06 07:34:12 [ 563 ] DEBUG : Stdout: PID TTY TIME CMD (cluster.py:121, run_and_check) 2026-05-06 07:34:12 [ 563 ] DEBUG : Stdout: 9 ? 00:00:03 clickhouse (cluster.py:121, run_and_check) 2026-05-06 07:34:12 [ 563 ] DEBUG : run container_id:roottestclusterdiscovery_gw8_node3_1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill clickhouse'] (cluster.py:2173, exec_in_container) 2026-05-06 07:34:12 [ 563 ] DEBUG : Command:['docker', 'exec', '-u', 'root', 'roottestclusterdiscovery_gw8_node3_1', 'bash', '-c', 'pkill clickhouse'] (cluster.py:113, run_and_check) 2026-05-06 07:34:12 [ 563 ] DEBUG : run container_id:roottestclusterdiscovery_gw8_node3_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2173, exec_in_container) 2026-05-06 07:34:12 [ 563 ] DEBUG : Command:['docker', 'exec', 'roottestclusterdiscovery_gw8_node3_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:113, run_and_check) 2026-05-06 07:34:13 [ 563 ] DEBUG : Stdout:9 (cluster.py:121, run_and_check) 2026-05-06 07:34:14 [ 563 ] DEBUG : run container_id:roottestclusterdiscovery_gw8_node3_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2173, exec_in_container) 2026-05-06 07:34:14 [ 563 ] DEBUG : Command:['docker', 'exec', 'roottestclusterdiscovery_gw8_node3_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:113, run_and_check) 2026-05-06 07:34:14 [ 563 ] DEBUG : Stdout:9 (cluster.py:121, run_and_check) 2026-05-06 07:34:15 [ 563 ] DEBUG : run container_id:roottestclusterdiscovery_gw8_node3_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2173, exec_in_container) 2026-05-06 07:34:15 [ 563 ] DEBUG : Command:['docker', 'exec', 'roottestclusterdiscovery_gw8_node3_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:113, run_and_check) 2026-05-06 07:34:15 [ 563 ] DEBUG : Stdout:9 (cluster.py:121, run_and_check) 2026-05-06 07:34:16 [ 563 ] DEBUG : run container_id:roottestclusterdiscovery_gw8_node3_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2173, exec_in_container) 2026-05-06 07:34:16 [ 563 ] DEBUG : Command:['docker', 'exec', 'roottestclusterdiscovery_gw8_node3_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:113, run_and_check) 2026-05-06 07:34:16 [ 563 ] DEBUG : Stdout:9 (cluster.py:121, run_and_check) 2026-05-06 07:34:17 [ 563 ] DEBUG : run container_id:roottestclusterdiscovery_gw8_node3_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2173, exec_in_container) 2026-05-06 07:34:17 [ 563 ] DEBUG : Command:['docker', 'exec', 'roottestclusterdiscovery_gw8_node3_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:113, run_and_check) 2026-05-06 07:34:17 [ 563 ] DEBUG : Stdout:9 (cluster.py:121, run_and_check) 2026-05-06 07:34:18 [ 563 ] DEBUG : run container_id:roottestclusterdiscovery_gw8_node3_1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2173, exec_in_container) 2026-05-06 07:34:18 [ 563 ] DEBUG : Command:['docker', 'exec', 'roottestclusterdiscovery_gw8_node3_1', 'bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:113, run_and_check) 2026-05-06 07:34:18 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node0 (cluster.py:3602, query) 2026-05-06 07:34:18 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node2 (cluster.py:3602, query) 2026-05-06 07:34:19 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node_observer (cluster.py:3602, query) 2026-05-06 07:34:21 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node0 (cluster.py:3602, query) 2026-05-06 07:34:21 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node2 (cluster.py:3602, query) 2026-05-06 07:34:21 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node_observer (cluster.py:3602, query) 2026-05-06 07:34:26 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node0 (cluster.py:3602, query) 2026-05-06 07:34:26 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node2 (cluster.py:3602, query) 2026-05-06 07:34:34 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node2 (cluster.py:3602, query) 2026-05-06 07:34:51 [ 563 ] DEBUG : Executing query SELECT count() FROM system.clusters WHERE cluster = 'test_auto_cluster' on node2 (cluster.py:3602, query) ---------------------------- Captured log teardown ----------------------------- 2026-05-06 07:34:52 [ 563 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/.env', '--project-name', 'roottestclusterdiscovery_gw8', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node0/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node1/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node3/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node4/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node_observer/docker-compose.yml', 'stop', '--timeout', '20'] (cluster.py:113, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Stderr:Stopping roottestclusterdiscovery_gw8_node2_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Stderr:Stopping roottestclusterdiscovery_gw8_node4_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Stderr:Stopping roottestclusterdiscovery_gw8_node1_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Stderr:Stopping roottestclusterdiscovery_gw8_node3_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Stderr:Stopping roottestclusterdiscovery_gw8_node0_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Stderr:Stopping roottestclusterdiscovery_gw8_node_observer_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Stderr:Stopping roottestclusterdiscovery_gw8_zoo2_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Stderr:Stopping roottestclusterdiscovery_gw8_zoo3_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Stderr:Stopping roottestclusterdiscovery_gw8_zoo1_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Stderr:Stopping roottestclusterdiscovery_gw8_node4_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Stderr:Stopping roottestclusterdiscovery_gw8_node1_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Stderr:Stopping roottestclusterdiscovery_gw8_node3_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Stderr:Stopping roottestclusterdiscovery_gw8_node2_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Stderr:Stopping roottestclusterdiscovery_gw8_node0_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Stderr:Stopping roottestclusterdiscovery_gw8_node_observer_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Stderr:Stopping roottestclusterdiscovery_gw8_zoo2_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Stderr:Stopping roottestclusterdiscovery_gw8_zoo3_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Stderr:Stopping roottestclusterdiscovery_gw8_zoo1_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node0/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node0/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node3/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node3/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node4/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node4/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node_observer/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node_observer/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-05-06 07:34:57 [ 563 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/.env', '--project-name', 'roottestclusterdiscovery_gw8', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node0/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node1/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node3/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node4/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_cluster_discovery/_instances_0_gw8/node_observer/docker-compose.yml', 'down', '--volumes'] (cluster.py:113, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Stderr:Removing roottestclusterdiscovery_gw8_node2_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Stderr:Removing roottestclusterdiscovery_gw8_node4_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Stderr:Removing roottestclusterdiscovery_gw8_node1_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Stderr:Removing roottestclusterdiscovery_gw8_node3_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Stderr:Removing roottestclusterdiscovery_gw8_node0_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Stderr:Removing roottestclusterdiscovery_gw8_node_observer_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Stderr:Removing roottestclusterdiscovery_gw8_zoo2_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Stderr:Removing roottestclusterdiscovery_gw8_zoo3_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Stderr:Removing roottestclusterdiscovery_gw8_zoo1_1 ... (cluster.py:123, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Stderr:Removing roottestclusterdiscovery_gw8_zoo1_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Stderr:Removing roottestclusterdiscovery_gw8_node_observer_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Stderr:Removing roottestclusterdiscovery_gw8_node4_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Stderr:Removing roottestclusterdiscovery_gw8_node3_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Stderr:Removing roottestclusterdiscovery_gw8_zoo2_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Stderr:Removing roottestclusterdiscovery_gw8_node2_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Stderr:Removing roottestclusterdiscovery_gw8_node0_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Stderr:Removing roottestclusterdiscovery_gw8_zoo3_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Stderr:Removing roottestclusterdiscovery_gw8_node1_1 ... done (cluster.py:123, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Stderr:Removing network roottestclusterdiscovery_gw8_default (cluster.py:123, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-05-06 07:34:58 [ 563 ] DEBUG : Docker networks for project roottestclusterdiscovery_gw8 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-05-06 07:34:58 [ 563 ] DEBUG : Docker containers for project roottestclusterdiscovery_gw8 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-05-06 07:34:58 [ 563 ] DEBUG : Docker volumes for project roottestclusterdiscovery_gw8 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-05-06 07:34:58 [ 563 ] DEBUG : Command:docker container list --all --filter name='^/roottestclusterdiscovery_gw8_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-05-06 07:34:58 [ 563 ] DEBUG : No running containers for project: roottestclusterdiscovery_gw8 (cluster.py:904, cleanup) 2026-05-06 07:34:58 [ 563 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-05-06 07:34:58 [ 563 ] DEBUG : Trying to prune unused images... (cluster.py:926, cleanup) 2026-05-06 07:34:58 [ 563 ] DEBUG : Command:['docker', 'image', 'prune', '-f'] (cluster.py:113, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:121, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Images pruned (cluster.py:929, cleanup) 2026-05-06 07:34:58 [ 563 ] DEBUG : Trying to prune unused volumes... (cluster.py:935, cleanup) 2026-05-06 07:34:58 [ 563 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-05-06 07:34:58 [ 563 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) ============================== slowest durations =============================== 133.19s call test_backward_compatibility/test_convert_ordinary.py::test_convert_ordinary_to_atomic 112.72s call test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated 105.16s call test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas 74.31s call test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_direct] 72.90s call test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_direct] 65.12s call test_cluster_discovery/test.py::test_cluster_discovery_startup_and_stop 45.59s call test_dictionaries_redis/test.py::test_redis_dictionaries[Float32_] 45.49s setup test_distributed_ddl/test_replicated_alter.py::test_replicated_alters[configs] 43.73s setup test_backup_restore_on_cluster/test_concurrency.py::test_kill_mutation_during_backup 42.61s setup test_backward_compatibility/test_aggregate_fixed_key.py::test_two_level_merge 41.94s call test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[cache] 41.25s call test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[cache] 40.90s setup test_distributed_ddl/test_replicated_alter.py::test_replicated_alters[configs_secure] 40.59s call test_distributed_ddl_parallel/test.py::test_smoke 40.17s call test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_hashed] 39.76s setup test_backward_compatibility/test.py::test_backward_compatability1 39.24s call test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_hashed] 39.04s call test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[direct] 38.99s call test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[direct] 37.18s call test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_cache] 36.51s call test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[flat] 36.37s call test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[hashed] 36.12s call test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_cache] 35.93s call test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[hashed] 35.90s call test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[flat] 34.42s teardown test_disable_insertion_and_mutation/test.py::test_disable_insertion_and_mutation 34.42s setup test_disable_insertion_and_mutation/test.py::test_disable_insertion_and_mutation 32.88s setup test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability 32.49s setup test_cluster_discovery/test.py::test_cluster_discovery_startup_and_stop 32.07s call test_distributed_ddl_parallel/test.py::test_smoke_parallel_dict_reload 30.29s setup test_distributed_ddl_parallel/test.py::test_all_in_parallel 29.83s call test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool 29.03s teardown test_backup_restore_on_cluster/test_concurrency.py::test_replicated_table 26.92s call test_distributed_ddl/test_replicated_alter.py::test_replicated_alters[configs] 26.05s call test_dictionaries_redis/test.py::test_redis_dictionaries[Int32_] 25.76s setup test_dictionaries_redis/test.py::test_redis_dictionaries[DateTime_] 25.27s call test_dictionaries_redis/test.py::test_redis_dictionaries[Float64_] 24.35s call test_config_xml_full/test.py::test_xml_full_conf 24.34s call test_backup_restore_on_cluster/test_concurrency.py::test_replicated_table 24.07s call test_dictionaries_redis/test.py::test_redis_dictionaries[Date_] 23.91s call test_dictionaries_redis/test.py::test_redis_dictionaries[Int16_] 22.02s call test_dictionaries_all_layouts_separate_sources/test_http.py::test_ranged[range_hashed] 21.34s call test_dictionaries_redis/test.py::test_redis_dictionaries[Int8_] 21.10s setup test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool 21.06s call test_dictionaries_redis/test.py::test_redis_dictionaries[Int64_] 20.88s call test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_ranged[range_hashed] 20.68s setup test_check_table/test.py::test_check_all_tables 20.12s call test_config_yaml_merge_keys/test.py::test_yaml_merge_keys_conf 20.11s call test_dictionaries_redis/test.py::test_redis_dictionaries[DateTime_] 20.03s setup test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_cache] 19.31s call test_dictionaries_redis/test.py::test_redis_dictionaries[String_] 19.14s call test_dictionaries_wait_for_load/test.py::test_wait_for_dictionaries_load 19.04s setup test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_cache] 18.94s setup test_ddl_config_hostname/test.py::test_ddl_queue_delete_add_replica 18.93s setup test_compression_nested_columns/test.py::test_nested_compression_codec 18.71s call test_dictionaries_redis/test.py::test_redis_dictionaries[UInt8_] 18.70s call test_dictionaries_redis/test.py::test_redis_dictionaries[UInt16_] 18.64s call test_dictionaries_redis/test.py::test_redis_dictionaries[UInt32_] 18.39s teardown test_backward_compatibility/test.py::test_backward_compatability1 18.35s call test_dictionaries_redis/test.py::test_redis_dictionaries[UUID_] 18.26s call test_dictionaries_redis/test.py::test_redis_dictionaries[UInt64_] 18.19s call test_distributed_ddl/test_replicated_alter.py::test_replicated_alters[configs_secure] 17.30s setup test_backward_compatibility/test_convert_ordinary.py::test_convert_ordinary_to_atomic 16.72s call test_config_decryption/test_wrong_settings.py::test_invalid_chars 16.08s setup test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[cluster_node1] 15.64s call test_distributed_ddl_parallel/test.py::test_two_in_parallel_two_queued 15.61s setup test_cluster_all_replicas/test.py::test_cluster 14.66s setup test_dictionaries_select_all/test.py::test_select_all[clickhouse_complex_integers_key_hashed] 14.40s call test_config_decryption/test_wrong_settings.py::test_subnodes 14.12s setup test_cleanup_dir_after_bad_zk_conn/test.py::test_cleanup_dir_after_wrong_replica_name 14.01s call test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability 13.98s setup test_dictionaries_wait_for_load/test.py::test_wait_for_dictionaries_load 13.78s teardown test_distributed_ddl/test_replicated_alter.py::test_replicated_alters[configs_secure] 13.76s setup test_config_corresponding_root/test.py::test_work 13.64s setup test_backward_compatibility/test_short_strings_aggregation.py::test_backward_compatability 11.71s call test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool 11.36s setup test_concurrent_queries_for_user_restriction/test.py::test_exception_message 11.26s call test_backup_restore_on_cluster/test_concurrency.py::test_kill_mutation_during_backup 11.20s call test_config_decryption/test_wrong_settings.py::test_wrong_method 11.05s call test_check_table/test.py::test_check_normal_table_corruption[] 11.05s teardown test_distributed_ddl_parallel/test.py::test_two_in_parallel_two_queued 10.97s setup test_config_reloader_interval/test.py::test_reload_config 10.86s teardown test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability 10.38s teardown test_distributed_async_insert_for_node_changes/test.py::test_distributed_async_insert_with_replica 10.15s teardown test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool 9.61s call test_check_table/test.py::test_check_replicated_table_corruption[-_0-.bin] 9.47s teardown test_check_table/test.py::test_check_replicated_table_simple[-_0] 9.43s teardown test_backward_compatibility/test_aggregate_fixed_key.py::test_two_level_merge 9.06s setup test_backward_compatibility/test_select_aggregate_alias_column.py::test_select_aggregate_alias_column 8.91s call test_config_decryption/test_wrong_settings.py::test_no_encryption_key 8.72s call test_check_table/test.py::test_check_all_tables 8.54s call test_distributed_ddl_parallel/test.py::test_all_in_parallel 8.47s teardown test_ddl_config_hostname/test.py::test_ddl_queue_delete_add_replica 8.28s call test_create_user_and_login/test.py::test_login_as_dropped_user_xml 7.92s teardown test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[hashed] 7.82s setup test_distributed_async_insert_for_node_changes/test.py::test_distributed_async_insert 7.78s setup test_custom_settings/test.py::test_custom_settings 7.76s call test_distributed_async_insert_for_node_changes/test.py::test_distributed_async_insert_with_replica 7.59s teardown test_backward_compatibility/test_short_strings_aggregation.py::test_backward_compatability 7.20s teardown test_cleanup_dir_after_bad_zk_conn/test.py::test_cleanup_dir_after_wrong_zk_path 7.19s call test_distributed_ddl_parallel/test.py::test_slow_dict_load_7 6.98s teardown test_cluster_discovery/test.py::test_cluster_discovery_startup_and_stop 6.78s call test_check_table/test.py::test_check_replicated_table_simple[-_0] 6.59s teardown test_dictionaries_redis/test.py::test_redis_dictionaries[UUID_] 6.47s teardown test_backward_compatibility/test_select_aggregate_alias_column.py::test_select_aggregate_alias_column 6.47s teardown test_compression_nested_columns/test.py::test_nested_compression_codec 6.46s call test_distributed_async_insert_for_node_changes/test.py::test_distributed_async_insert 6.43s teardown test_custom_settings/test.py::test_illformed_setting 6.39s teardown test_cluster_all_replicas/test.py::test_skip_unavailable_replica[two_shards_three_nodes] 6.19s teardown test_create_user_and_login/test.py::test_login_as_dropped_user_xml 6.01s setup test_create_query_constraints/test.py::test_create_query_const_constraints 5.98s setup test_create_user_and_login/test.py::test_grant_create_user 5.91s teardown test_concurrent_queries_for_user_restriction/test.py::test_exception_message 5.86s teardown test_dictionaries_wait_for_load/test.py::test_wait_for_dictionaries_load 5.63s teardown test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[hashed] 5.56s call test_create_query_constraints/test.py::test_create_query_minmax_constraints 5.47s call test_cleanup_dir_after_bad_zk_conn/test.py::test_cleanup_dir_after_wrong_zk_path 5.42s call test_disable_insertion_and_mutation/test.py::test_disable_insertion_and_mutation 5.22s call test_backward_compatibility/test.py::test_backward_compatability1 4.78s call test_distributed_ddl_parallel/test.py::test_smoke_parallel 4.66s teardown test_config_reloader_interval/test.py::test_reload_config 4.65s teardown test_backward_compatibility/test_convert_ordinary.py::test_convert_ordinary_to_atomic 4.64s teardown test_create_query_constraints/test.py::test_create_query_minmax_constraints 4.29s teardown test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_complex_mixed_key_cache] 3.69s call test_ddl_config_hostname/test.py::test_ddl_queue_delete_add_replica 3.37s call test_compression_nested_columns/test.py::test_nested_compression_codec 3.00s call test_backward_compatibility/test_aggregate_fixed_key.py::test_two_level_merge 2.93s call test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_complex_mixed_key_cache] 2.60s call test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_cache] 2.59s call test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_complex_integers_key_cache] 2.53s call test_backward_compatibility/test_short_strings_aggregation.py::test_backward_compatability 2.52s call test_create_query_constraints/test.py::test_create_query_const_constraints 2.32s call test_backward_compatibility/test_select_aggregate_alias_column.py::test_select_aggregate_alias_column 1.99s call test_create_user_and_login/test.py::test_login_as_dropped_user 1.46s call test_concurrent_queries_for_user_restriction/test.py::test_exception_message 1.40s call test_config_reloader_interval/test.py::test_reload_config 1.32s call test_custom_settings/test.py::test_custom_settings 1.23s call test_create_user_and_login/test.py::test_grant_create_user 0.92s call test_cluster_all_replicas/test.py::test_cluster 0.91s call test_create_user_and_login/test.py::test_login 0.84s teardown test_backup_restore_on_cluster/test_concurrency.py::test_kill_mutation_during_backup 0.75s call test_cleanup_dir_after_bad_zk_conn/test.py::test_cleanup_dir_after_wrong_replica_name 0.70s call test_dictionaries_select_all/test.py::test_select_all[clickhouse_complex_integers_key_hashed] 0.70s call test_cluster_all_replicas/test.py::test_global_in 0.65s call test_dictionaries_select_all/test.py::test_select_all[clickhouse_flat] 0.65s call test_dictionaries_select_all/test.py::test_select_all[clickhouse_hashed] 0.60s call test_dictionaries_select_all/test.py::test_select_all[clickhouse_range_hashed] 0.60s call test_dictionaries_select_all/test.py::test_select_all[clickhouse_complex_mixed_key_hashed] 0.52s call test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[cluster_node2] 0.42s call test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[cluster_node1] 0.40s call test_custom_settings/test.py::test_illformed_setting 0.32s call test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[dod_node1] 0.32s call test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_group_by_key[cluster_node2] 0.27s call test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[dod_node2] 0.27s call test_cluster_all_replicas/test.py::test_skip_unavailable_replica[one_shard_three_nodes] 0.27s call test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_group_by_key[cluster_node1] 0.22s teardown test_create_user_and_login/test.py::test_login_as_dropped_user 0.22s teardown test_create_user_and_login/test.py::test_login 0.22s call test_cluster_all_replicas/test.py::test_error_on_unavailable_replica[one_shard_three_nodes] 0.22s call test_cluster_all_replicas/test.py::test_skip_unavailable_replica[two_shards_three_nodes] 0.22s call test_cluster_all_replicas/test.py::test_error_on_unavailable_replica[two_shards_three_nodes] 0.17s teardown test_create_user_and_login/test.py::test_grant_create_user 0.06s setup test_config_decryption/test_wrong_settings.py::test_invalid_chars 0.01s teardown test_distributed_ddl/test_replicated_alter.py::test_replicated_alters[configs] 0.01s setup test_distributed_ddl_parallel/test.py::test_smoke_parallel_dict_reload 0.01s teardown test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool 0.01s teardown test_config_xml_full/test.py::test_xml_full_conf 0.01s teardown test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_cache] 0.00s teardown test_distributed_ddl_parallel/test.py::test_smoke 0.00s setup test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_ranged[range_hashed] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_direct] 0.00s setup test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_hashed] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_cache] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_direct] 0.00s setup test_distributed_ddl_parallel/test.py::test_two_in_parallel_two_queued 0.00s setup test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[cache] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_http.py::test_ranged[range_hashed] 0.00s teardown test_dictionaries_redis/test.py::test_redis_dictionaries[Float32_] 0.00s setup test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[cache] 0.00s setup test_dictionaries_redis/test.py::test_redis_dictionaries[Int64_] 0.00s teardown test_dictionaries_redis/test.py::test_redis_dictionaries[Int32_] 0.00s setup test_dictionaries_redis/test.py::test_redis_dictionaries[Float64_] 0.00s setup test_distributed_ddl_parallel/test.py::test_smoke_parallel 0.00s teardown test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated 0.00s setup test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_direct] 0.00s setup test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_hashed] 0.00s teardown test_dictionaries_redis/test.py::test_redis_dictionaries[Date_] 0.00s setup test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool 0.00s setup test_backup_restore_on_cluster/test_concurrency.py::test_replicated_table 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_hashed] 0.00s setup test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas 0.00s teardown test_distributed_ddl_parallel/test.py::test_slow_dict_load_7 0.00s setup test_dictionaries_redis/test.py::test_redis_dictionaries[Int16_] 0.00s teardown test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas 0.00s setup test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_direct] 0.00s teardown test_config_yaml_merge_keys/test.py::test_yaml_merge_keys_conf 0.00s teardown test_distributed_ddl_parallel/test.py::test_smoke_parallel_dict_reload 0.00s setup test_check_table/test.py::test_check_replicated_table_corruption[-_0-.bin] 0.00s setup test_dictionaries_redis/test.py::test_redis_dictionaries[Float32_] 0.00s teardown test_distributed_ddl_parallel/test.py::test_smoke_parallel 0.00s setup test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_group_by_key[cluster_node1] 0.00s setup test_dictionaries_all_layouts_separate_sources/test_http.py::test_ranged[range_hashed] 0.00s setup test_config_xml_full/test.py::test_xml_full_conf 0.00s setup test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[hashed] 0.00s teardown test_distributed_ddl_parallel/test.py::test_all_in_parallel 0.00s setup test_config_decryption/test_wrong_settings.py::test_wrong_method 0.00s setup test_check_table/test.py::test_check_replicated_table_simple[-_0] 0.00s setup test_cluster_all_replicas/test.py::test_skip_unavailable_replica[one_shard_three_nodes] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_hashed] 0.00s teardown test_dictionaries_redis/test.py::test_redis_dictionaries[Float64_] 0.00s setup test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_ranged[range_hashed] 0.00s setup test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[dod_node2] 0.00s teardown test_dictionaries_select_all/test.py::test_select_all[clickhouse_complex_mixed_key_hashed] 0.00s setup test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[direct] 0.00s setup test_dictionaries_select_all/test.py::test_select_all[clickhouse_hashed] 0.00s setup test_config_yaml_merge_keys/test.py::test_yaml_merge_keys_conf 0.00s teardown test_dictionaries_redis/test.py::test_redis_dictionaries[Int8_] 0.00s setup test_dictionaries_redis/test.py::test_redis_dictionaries[UInt16_] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[flat] 0.00s teardown test_dictionaries_redis/test.py::test_redis_dictionaries[DateTime_] 0.00s teardown test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[dod_node2] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[cache] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[direct] 0.00s teardown test_cluster_all_replicas/test.py::test_global_in 0.00s teardown test_check_table/test.py::test_check_all_tables 0.00s setup test_config_decryption/test_wrong_settings.py::test_subnodes 0.00s setup test_create_user_and_login/test.py::test_login 0.00s teardown test_dictionaries_select_all/test.py::test_select_all[clickhouse_flat] 0.00s setup test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[direct] 0.00s teardown test_cleanup_dir_after_bad_zk_conn/test.py::test_cleanup_dir_after_wrong_replica_name 0.00s teardown test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[cluster_node1] 0.00s setup test_create_user_and_login/test.py::test_login_as_dropped_user 0.00s teardown test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_group_by_key[cluster_node2] 0.00s setup test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[cluster_node2] 0.00s setup test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[flat] 0.00s teardown test_dictionaries_redis/test.py::test_redis_dictionaries[String_] 0.00s setup test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_complex_integers_key_cache] 0.00s setup test_dictionaries_select_all/test.py::test_select_all[clickhouse_range_hashed] 0.00s teardown test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[dod_node1] 0.00s setup test_dictionaries_redis/test.py::test_redis_dictionaries[UInt64_] 0.00s setup test_cluster_all_replicas/test.py::test_error_on_unavailable_replica[one_shard_three_nodes] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[flat] 0.00s teardown test_check_table/test.py::test_check_replicated_table_corruption[-_0-.bin] 0.00s setup test_dictionaries_redis/test.py::test_redis_dictionaries[UInt8_] 0.00s setup test_dictionaries_redis/test.py::test_redis_dictionaries[Int32_] 0.00s setup test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_group_by_key[cluster_node2] 0.00s teardown test_config_decryption/test_wrong_settings.py::test_no_encryption_key 0.00s teardown test_dictionaries_redis/test.py::test_redis_dictionaries[Int16_] 0.00s setup test_dictionaries_redis/test.py::test_redis_dictionaries[UInt32_] 0.00s setup test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[hashed] 0.00s teardown test_create_query_constraints/test.py::test_create_query_const_constraints 0.00s setup test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[dod_node1] 0.00s setup test_dictionaries_redis/test.py::test_redis_dictionaries[Int8_] 0.00s setup test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_cache] 0.00s teardown test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_cache] 0.00s setup test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[flat] 0.00s setup test_check_table/test.py::test_check_normal_table_corruption[] 0.00s setup test_dictionaries_redis/test.py::test_redis_dictionaries[String_] 0.00s teardown test_dictionaries_redis/test.py::test_redis_dictionaries[UInt32_] 0.00s setup test_custom_settings/test.py::test_illformed_setting 0.00s setup test_create_query_constraints/test.py::test_create_query_minmax_constraints 0.00s teardown test_cluster_all_replicas/test.py::test_cluster 0.00s teardown test_custom_settings/test.py::test_custom_settings 0.00s setup test_distributed_ddl_parallel/test.py::test_slow_dict_load_7 0.00s setup test_cluster_all_replicas/test.py::test_skip_unavailable_replica[two_shards_three_nodes] 0.00s setup test_distributed_ddl_parallel/test.py::test_smoke 0.00s setup test_cluster_all_replicas/test.py::test_error_on_unavailable_replica[two_shards_three_nodes] 0.00s teardown test_dictionaries_select_all/test.py::test_select_all[clickhouse_hashed] 0.00s setup test_dictionaries_redis/test.py::test_redis_dictionaries[UUID_] 0.00s teardown test_dictionaries_redis/test.py::test_redis_dictionaries[UInt16_] 0.00s teardown test_dictionaries_redis/test.py::test_redis_dictionaries[UInt64_] 0.00s setup test_cleanup_dir_after_bad_zk_conn/test.py::test_cleanup_dir_after_wrong_zk_path 0.00s teardown test_config_decryption/test_wrong_settings.py::test_invalid_chars 0.00s setup test_create_user_and_login/test.py::test_login_as_dropped_user_xml 0.00s setup test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_complex_mixed_key_cache] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[cache] 0.00s teardown test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_group_by_key[cluster_node1] 0.00s teardown test_check_table/test.py::test_check_normal_table_corruption[] 0.00s setup test_dictionaries_redis/test.py::test_redis_dictionaries[Date_] 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[direct] 0.00s setup test_dictionaries_select_all/test.py::test_select_all[clickhouse_complex_mixed_key_hashed] 0.00s call test_config_corresponding_root/test.py::test_work 0.00s teardown test_distributed_async_insert_for_node_changes/test.py::test_distributed_async_insert 0.00s teardown test_dictionaries_redis/test.py::test_redis_dictionaries[Int64_] 0.00s setup test_dictionaries_select_all/test.py::test_select_all[clickhouse_flat] 0.00s teardown test_config_decryption/test_wrong_settings.py::test_wrong_method 0.00s teardown test_dictionaries_select_all/test.py::test_select_all[clickhouse_complex_integers_key_hashed] 0.00s teardown test_config_decryption/test_wrong_settings.py::test_subnodes 0.00s teardown test_config_corresponding_root/test.py::test_work 0.00s teardown test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[cluster_node2] 0.00s teardown test_dictionaries_redis/test.py::test_redis_dictionaries[UInt8_] 0.00s teardown test_cluster_all_replicas/test.py::test_error_on_unavailable_replica[one_shard_three_nodes] 0.00s teardown test_dictionaries_select_all/test.py::test_select_all[clickhouse_range_hashed] 0.00s setup test_config_decryption/test_wrong_settings.py::test_no_encryption_key 0.00s setup test_cluster_all_replicas/test.py::test_global_in 0.00s teardown test_cluster_all_replicas/test.py::test_skip_unavailable_replica[one_shard_three_nodes] 0.00s setup test_distributed_async_insert_for_node_changes/test.py::test_distributed_async_insert_with_replica 0.00s teardown test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_complex_integers_key_cache] 0.00s teardown test_cluster_all_replicas/test.py::test_error_on_unavailable_replica[two_shards_three_nodes] =========================== short test summary info ============================ FAILED test_cluster_discovery/test.py::test_cluster_discovery_startup_and_stop PASSED test_dictionaries_select_all/test.py::test_select_all[clickhouse_complex_integers_key_hashed] PASSED test_dictionaries_select_all/test.py::test_select_all[clickhouse_complex_mixed_key_hashed] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[cluster_node1] PASSED test_cluster_all_replicas/test.py::test_cluster PASSED test_dictionaries_select_all/test.py::test_select_all[clickhouse_flat] PASSED test_cluster_all_replicas/test.py::test_error_on_unavailable_replica[one_shard_three_nodes] PASSED test_config_decryption/test_wrong_settings.py::test_invalid_chars PASSED test_cluster_all_replicas/test.py::test_error_on_unavailable_replica[two_shards_three_nodes] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[cluster_node2] PASSED test_dictionaries_select_all/test.py::test_select_all[clickhouse_hashed] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[dod_node1] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[dod_node2] PASSED test_cluster_all_replicas/test.py::test_global_in PASSED test_dictionaries_select_all/test.py::test_select_all[clickhouse_range_hashed] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_group_by_key[cluster_node1] PASSED test_cluster_all_replicas/test.py::test_skip_unavailable_replica[one_shard_three_nodes] PASSED test_cluster_all_replicas/test.py::test_skip_unavailable_replica[two_shards_three_nodes] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_group_by_key[cluster_node2] PASSED test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_cache] PASSED test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_complex_integers_key_cache] PASSED test_create_user_and_login/test.py::test_grant_create_user PASSED test_config_decryption/test_wrong_settings.py::test_no_encryption_key PASSED test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_complex_mixed_key_cache] PASSED test_create_user_and_login/test.py::test_login PASSED test_create_user_and_login/test.py::test_login_as_dropped_user PASSED test_check_table/test.py::test_check_all_tables PASSED test_create_user_and_login/test.py::test_login_as_dropped_user_xml PASSED test_distributed_ddl_parallel/test.py::test_all_in_parallel PASSED test_config_decryption/test_wrong_settings.py::test_subnodes PASSED test_check_table/test.py::test_check_normal_table_corruption[] PASSED test_cleanup_dir_after_bad_zk_conn/test.py::test_cleanup_dir_after_wrong_replica_name PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[DateTime_] PASSED test_distributed_ddl_parallel/test.py::test_slow_dict_load_7 PASSED test_check_table/test.py::test_check_replicated_table_corruption[-_0-.bin] PASSED test_cleanup_dir_after_bad_zk_conn/test.py::test_cleanup_dir_after_wrong_zk_path PASSED test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool PASSED test_config_decryption/test_wrong_settings.py::test_wrong_method PASSED test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_cache] PASSED test_check_table/test.py::test_check_replicated_table_simple[-_0] PASSED test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_cache] PASSED test_distributed_async_insert_for_node_changes/test.py::test_distributed_async_insert PASSED test_custom_settings/test.py::test_custom_settings PASSED test_custom_settings/test.py::test_illformed_setting PASSED test_distributed_async_insert_for_node_changes/test.py::test_distributed_async_insert_with_replica PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[Date_] PASSED test_backup_restore_on_cluster/test_concurrency.py::test_kill_mutation_during_backup PASSED test_distributed_ddl_parallel/test.py::test_smoke PASSED test_distributed_ddl_parallel/test.py::test_smoke_parallel PASSED test_backup_restore_on_cluster/test_concurrency.py::test_replicated_table PASSED test_backward_compatibility/test.py::test_backward_compatability1 PASSED test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[Float32_] PASSED test_backward_compatibility/test_aggregate_fixed_key.py::test_two_level_merge PASSED test_distributed_ddl_parallel/test.py::test_smoke_parallel_dict_reload PASSED test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_direct] PASSED test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_direct] PASSED test_distributed_ddl/test_replicated_alter.py::test_replicated_alters[configs] PASSED test_distributed_ddl_parallel/test.py::test_two_in_parallel_two_queued PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[Float64_] PASSED test_backward_compatibility/test_select_aggregate_alias_column.py::test_select_aggregate_alias_column PASSED test_create_query_constraints/test.py::test_create_query_const_constraints PASSED test_create_query_constraints/test.py::test_create_query_minmax_constraints PASSED test_config_corresponding_root/test.py::test_work PASSED test_config_reloader_interval/test.py::test_reload_config PASSED test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[Int16_] PASSED test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_hashed] PASSED test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_hashed] PASSED test_distributed_ddl/test_replicated_alter.py::test_replicated_alters[configs_secure] PASSED test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_ranged[range_hashed] PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[Int32_] PASSED test_config_xml_full/test.py::test_xml_full_conf PASSED test_dictionaries_all_layouts_separate_sources/test_http.py::test_ranged[range_hashed] PASSED test_disable_insertion_and_mutation/test.py::test_disable_insertion_and_mutation PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[Int64_] PASSED test_backward_compatibility/test_short_strings_aggregation.py::test_backward_compatability PASSED test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[cache] PASSED test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[cache] PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[Int8_] PASSED test_concurrent_queries_for_user_restriction/test.py::test_exception_message PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[String_] PASSED test_dictionaries_wait_for_load/test.py::test_wait_for_dictionaries_load PASSED test_config_yaml_merge_keys/test.py::test_yaml_merge_keys_conf PASSED test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas PASSED test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[direct] PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[UInt16_] PASSED test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[direct] PASSED test_backward_compatibility/test_convert_ordinary.py::test_convert_ordinary_to_atomic PASSED test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[UInt32_] PASSED test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[flat] PASSED test_compression_nested_columns/test.py::test_nested_compression_codec PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[UInt64_] PASSED test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[flat] PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[UInt8_] PASSED test_ddl_config_hostname/test.py::test_ddl_queue_delete_add_replica PASSED test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[hashed] PASSED test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[hashed] PASSED test_dictionaries_redis/test.py::test_redis_dictionaries[UUID_] =================== 1 failed, 99 passed in 365.16s (0:06:05) =================== Traceback (most recent call last): File "/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration/./runner", line 445, in subprocess.check_call(cmd, shell=True) File "/usr/lib/python3.10/subprocess.py", line 369, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'docker run --rm --name clickhouse_integration_tests_g2b5um --privileged --dns-search='.' --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=2cffe1eae894 -e DOCKER_BASE_TAG=1e0b53d756cf -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=caad4729259e -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 test_backup_restore_on_cluster/test_concurrency.py::test_kill_mutation_during_backup test_backup_restore_on_cluster/test_concurrency.py::test_replicated_table test_backward_compatibility/test.py::test_backward_compatability1 test_backward_compatibility/test_aggregate_fixed_key.py::test_two_level_merge test_backward_compatibility/test_convert_ordinary.py::test_convert_ordinary_to_atomic test_backward_compatibility/test_memory_bound_aggregation.py::test_backward_compatability test_backward_compatibility/test_select_aggregate_alias_column.py::test_select_aggregate_alias_column test_backward_compatibility/test_short_strings_aggregation.py::test_backward_compatability test_check_table/test.py::test_check_all_tables 'test_check_table/test.py::test_check_normal_table_corruption[]' 'test_check_table/test.py::test_check_replicated_table_corruption[-_0-.bin]' 'test_check_table/test.py::test_check_replicated_table_simple[-_0]' test_cleanup_dir_after_bad_zk_conn/test.py::test_cleanup_dir_after_wrong_replica_name test_cleanup_dir_after_bad_zk_conn/test.py::test_cleanup_dir_after_wrong_zk_path test_cluster_all_replicas/test.py::test_cluster 'test_cluster_all_replicas/test.py::test_error_on_unavailable_replica[one_shard_three_nodes]' 'test_cluster_all_replicas/test.py::test_error_on_unavailable_replica[two_shards_three_nodes]' test_cluster_all_replicas/test.py::test_global_in 'test_cluster_all_replicas/test.py::test_skip_unavailable_replica[one_shard_three_nodes]' 'test_cluster_all_replicas/test.py::test_skip_unavailable_replica[two_shards_three_nodes]' test_cluster_discovery/test.py::test_cluster_discovery_startup_and_stop test_compression_nested_columns/test.py::test_nested_compression_codec test_concurrent_queries_for_user_restriction/test.py::test_exception_message test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_in_empty_pool_replicated test_concurrent_ttl_merges/test.py::test_limited_ttl_merges_two_replicas test_concurrent_ttl_merges/test.py::test_no_ttl_merges_in_busy_pool test_config_corresponding_root/test.py::test_work test_config_decryption/test_wrong_settings.py::test_invalid_chars test_config_decryption/test_wrong_settings.py::test_no_encryption_key test_config_decryption/test_wrong_settings.py::test_subnodes test_config_decryption/test_wrong_settings.py::test_wrong_method test_config_reloader_interval/test.py::test_reload_config test_config_xml_full/test.py::test_xml_full_conf test_config_yaml_merge_keys/test.py::test_yaml_merge_keys_conf test_create_query_constraints/test.py::test_create_query_const_constraints test_create_query_constraints/test.py::test_create_query_minmax_constraints test_create_user_and_login/test.py::test_grant_create_user test_create_user_and_login/test.py::test_login test_create_user_and_login/test.py::test_login_as_dropped_user test_create_user_and_login/test.py::test_login_as_dropped_user_xml test_custom_settings/test.py::test_custom_settings test_custom_settings/test.py::test_illformed_setting test_ddl_config_hostname/test.py::test_ddl_queue_delete_add_replica 'test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_cache]' 'test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_direct]' 'test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_complex[complex_key_hashed]' 'test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_ranged[range_hashed]' 'test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[cache]' 'test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[direct]' 'test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[flat]' 'test_dictionaries_all_layouts_separate_sources/test_clickhouse_remote.py::test_simple[hashed]' 'test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_cache]' 'test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_direct]' 'test_dictionaries_all_layouts_separate_sources/test_http.py::test_complex[complex_key_hashed]' 'test_dictionaries_all_layouts_separate_sources/test_http.py::test_ranged[range_hashed]' 'test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[cache]' 'test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[direct]' 'test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[flat]' 'test_dictionaries_all_layouts_separate_sources/test_http.py::test_simple[hashed]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[DateTime_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[Date_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[Float32_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[Float64_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[Int16_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[Int32_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[Int64_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[Int8_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[String_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[UInt16_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[UInt32_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[UInt64_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[UInt8_]' 'test_dictionaries_redis/test.py::test_redis_dictionaries[UUID_]' 'test_dictionaries_select_all/test.py::test_select_all[clickhouse_complex_integers_key_hashed]' 'test_dictionaries_select_all/test.py::test_select_all[clickhouse_complex_mixed_key_hashed]' 'test_dictionaries_select_all/test.py::test_select_all[clickhouse_flat]' 'test_dictionaries_select_all/test.py::test_select_all[clickhouse_hashed]' 'test_dictionaries_select_all/test.py::test_select_all[clickhouse_range_hashed]' 'test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_cache]' 'test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_complex_integers_key_cache]' 'test_dictionaries_select_all/test.py::test_select_all_from_cached[clickhouse_complex_mixed_key_cache]' test_dictionaries_wait_for_load/test.py::test_wait_for_dictionaries_load test_disable_insertion_and_mutation/test.py::test_disable_insertion_and_mutation test_distributed_async_insert_for_node_changes/test.py::test_distributed_async_insert test_distributed_async_insert_for_node_changes/test.py::test_distributed_async_insert_with_replica 'test_distributed_ddl/test_replicated_alter.py::test_replicated_alters[configs]' 'test_distributed_ddl/test_replicated_alter.py::test_replicated_alters[configs_secure]' test_distributed_ddl_parallel/test.py::test_all_in_parallel test_distributed_ddl_parallel/test.py::test_slow_dict_load_7 test_distributed_ddl_parallel/test.py::test_smoke test_distributed_ddl_parallel/test.py::test_smoke_parallel test_distributed_ddl_parallel/test.py::test_smoke_parallel_dict_reload test_distributed_ddl_parallel/test.py::test_two_in_parallel_two_queued 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[cluster_node1]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[cluster_node2]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[dod_node1]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_sum[dod_node2]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_group_by_key[cluster_node1]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_group_by_key[cluster_node2]' -vvv" altinityinfra/integration-tests-runner:37a9815fd2fa ' returned non-zero exit status 1.